Remote control system, remote control method and program

ABSTRACT

A control target device  30  is, for example, an agricultural machine for performing various processes such as watering, additional fertilization, agrochemical spreading, and harvesting on a plant grown in a field. A moving image-capturing device  20  is a device that is movable and has an image-capturing function, and is, for example, a rotary-wing aircraft called a drone or a multicopter. A remote control device  10  analyzes an image of the field captured by the moving image-capturing device  30,  determines what process should be performed on the plant according to the analysis result, and instructs the control target device  30  to perform the determined process.

TECHNICAL FIELD

The present invention relates to a remote control system, a remotecontrol method and a program.

BACKGROUND ART

Various techniques have been developed to support human work. Forexample, as a technique for supporting agricultural work, PatentDocument 1 a system that includes a plurality of agricultural machines 2which are movable and perform agricultural work, a data collectingdevice 3 which collects agricultural work data in each agriculturalmachine 2 and transmits the data to the outside, and a plurality ofmovable portable terminals 4 which receive the agricultural work datatransmitted from the collecting device 3 and transmit the data to theserver 5. The agricultural work data accumulated in the server 5 is usedby an agricultural worker to create a management plan by creating adaily work report, making a work plan, or graphing the agricultural workresult of each field.

PRIOR ART DOCUMENT Patent Document

Patent Document 1: Japanese Patent Application Publication No.2014-194601

SUMMARY OF THE INVENTION Technical Problem

For example, in the technology described in Patent Document 1, it isentrusted to the agricultural worker to determine what kind ofagricultural work to be performed and when. On the other hand, an objectof the present invention is to make it possible to determine whether ornot to perform various processes, for example, such as agricultural workwithout entrusting it to humans.

Technical Solution

The present invention provides a remote control system including anacquisition unit that acquires image data indicating an image capturedby an image-capturing device and position data indicating a position ofthe image, a storage unit that stores a condition regarding an analysisresult of the image data and a process to be performed by a controltarget device in association with each other, a determination unit thatanalyzes the acquired image data and determines whether the analysisresult matches the stored condition, and an instruction unit that, whenthe analysis result matches the stored condition, instructs the controltarget device to perform the process associated with the condition in anarea based on the acquired position data.

The image-capturing device may be a moving device that is movable.

The remote control system may further include a movement control unitthat controls movement of the image-capturing device.

The movement control unit may control the movement of theimage-capturing device in accordance with a type of an animal or plantincluded in the area where the process is performed.

The movement control unit may control the movement of theimage-capturing device in accordance with the process, and the analysisunit may determine whether the analysis result matches the conditionthat is stored in association with the process.

The control target device may be the moving device.

The area based on the acquired position data may be an area that doesnot include the position indicated by the position data, and may be anarea having a predetermined positional relationship with the positionindicated by the position data.

The remote control system may further include a history storage unitthat stores, as a process history, the instructed process and the areawhere the process has been performed, and the instruction unit maydetermine a process to be instructed to the control target device basedon the stored processing history.

The instruction unit may determine a process to be instructed to thecontrol target device based on information related to a naturalenvironment at the position of the image before a time when the image iscaptured.

Further, the present invention provides a remote control methodincluding acquiring image data indicating an image captured by animage-capturing device and position data indicating a position of theimage, in a storage unit that stores a condition regarding an analysisresult of the image data and a process to be performed by a controltarget device in association with each other, determining whether ananalysis result of the acquired image data matches the stored condition,and when the analysis result matches the stored condition, instructingthe control target device to perform the process associated with thecondition in an area based on the acquired position data.

Furthermore, the present invention provides a program for causing one ormore computers to execute acquiring image data indicating an imagecaptured by an image-capturing device and position data indicating aposition of the image, in a storage unit that stores a conditionregarding an analysis result of the image data and a process to beperformed by a control target device in association with each other,determining whether an analysis result of the acquired image datamatches the stored condition, and when the analysis result matches thestored condition, instructing the control target device to perform theprocess associated with the condition in an area based on the acquiredposition data.

Effects of the Invention

According to the present invention, it is possible to determine whetheror not to perform a process without entrusting it to humans.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram exemplifying an overview of a remote control system1 according to an embodiment of the present invention.

FIG. 2 is a diagram exemplifying a hardware configuration of a movingimage-capturing device 20.

FIG. 3 is a diagram exemplifying a hardware configuration of a remotecontrol device 10.

FIG. 4 is a diagram exemplifying information stored in an auxiliarystorage device 104 of a remote control device 10.

FIG. 5 is a diagram exemplifying a condition process table stored in anauxiliary storage device 104 of a remote control device 10.

FIG. 6 is a diagram exemplifying a functional configuration of a remotecontrol system 1.

FIG. 7 is a sequence chart exemplifying an operation of a remote controlsystem 1.

DESCRIPTION OF REFERENCE NUMBERS

1: remote control system, 10: remote control device, 11: movementcontrol unit, 12: acquisition unit, 13: storage unit, 14: determinationunit, 15: instruction unit, 16: history storage unit, 20: movingimage-capturing device, 21: movement unit, 22: image-capturing unit, 23:transmission unit, 24: position detection unit, 30: control targetdevice, 31: processing unit, 90: network, 101: CPU, 102: RAM, 103: ROM,104: auxiliary storage device, 105: communication IF, 201: CPU, 202:RAM, 203: ROM, 204: auxiliary storage device, 205: communication IF,206: camera, 207: sensor, 2081: rotation mechanism.

DETAILED DESCRIPTION OF THE EMBODIMENTS 1. CONFIGURATION

FIG. 1 is a diagram exemplifying an overview of a remote control system1 according to an embodiment of the present invention. The remotecontrol system 1 includes a remote control device 10, a movingimage-capturing device 20, a control target device 30, and a network 90communicably connecting these devices 10 to 30. The network 90 is, forexample, a LAN (Local Area Network), a WAN (Wide Area Network), or acombination thereof, and may include a wired section or a wirelesssection. The control target device 30 is, for example, an agriculturalmachine for performing various processes such as watering, additionalfertilization, agrochemical spreading, and harvesting on a plant grownin a field. The moving image-capturing device 30 is a device that ismovable and has an image-capturing function, and is a rotary-wingaircraft called a drone or a multicopter in the present embodiment. Themoving image-capturing device 30 flies above the field according to aninstruction of the remote control device 10, and captures an image of aplant cultivated in the field. The remote control device 10 analyzes theimage captured by the moving image-capturing device 30, determines whatprocess should be performed on the plant according to the analysisresult, and instructs the control target device 30 to perform thedetermined process. The control target device 30 performs the instructedprocess automatically or in accordance with an operation of anagricultural worker.

FIG. 2 is a diagram exemplifying a hardware configuration of a movingimage-capturing device 20. The moving image-capturing device 20 includesa CPU 201 (Central Processing Unit), a ROM (Read Only Memory) 202, a RAM(Random Access Memory) 203, an auxiliary storage device 204, acommunication IF 205, a camera 206, a sensor 207, and a rotationmechanism 208. The CPU 201 is a processor that performs variousoperations. The RAM 202 is a volatile memory that functions as a workarea when the CPU 201 executes a program. The ROM 203 is, for example, anon-volatile memory that stores a program and data used for starting themoving image-capturing device 20. The auxiliary storage device 204 is,for example, a non-volatile storage device such as a hard disk drive(HDD) or a solid state drive (SSD) and stores a program and data used inthe moving image-capturing device 20. The CPU 201 executes the programso that the moving image-capturing device 20 functions as a computerdevice and functions shown in FIG. 6 to be described below areimplemented. The communication IF 205 is an interface that performscommunication via the network 90 in accordance with a predeterminedcommunication standard. The communication standard may be a wirelesscommunication standard or a wired communication standard. The camera 206captures an image of, for example, a space vertically below the movingimage-capturing device 20, and generates image data indicating thecaptured image. The positioning device 207 is, for example, a GPS(Global Positioning System) unit, and detects a position of the movingimage-capturing device 20. The rotation mechanism 208 is means forgenerating buoyancy in the moving image-capturing device 20, andincludes a propeller, a shaft, a motor, and other driving mechanisms.

FIG. 3 is a diagram exemplifying a hardware configuration of a remotecontrol device 10. The remote control device 10 is a computer deviceincluding a CPU 101, a ROM 102, a RAM 103, an auxiliary storage device104, and a communication IF 105. The CPU 101 is a processor thatperforms various operations. The RAM 102 is a volatile memory thatfunctions as a work area when the CPU 101 executes a program. The ROM103 is a non-volatile memory that stores, for example, a program anddata used for starting the remote control device 10. The auxiliarystorage device 104 is, for example, a non-volatile storage device suchas an HDD or an SSD and stores a program and data used in the remotecontrol device 10. The CPU 101 executes the program thereby implementingfunctions shown in FIG. 6 to be described below. The communication IF105 is an interface that performs communication via a network 90 inaccordance with a predetermined communication standard.

FIG. 4 is a diagram exemplifying information stored in an auxiliarystorage device 104 of a remote control device 10. The auxiliary storagedevice 104 stores a date and time when an image (here, a moving image)is captured by a moving image-capturing device 10, position dataindicating a position at which the image is captured, image dataindicating the captured image, and a result of analyzing the capturedimage, a processing target position data indicating a position of anarea in which a process is required as the analyzing result, an analysisresult of analyzing the captured image by an image recognitiontechnology, and a content of the required process in association witheach other.

FIG. 5 is a diagram exemplifying a condition process table stored in anauxiliary storage device 104 of a remote control device 10. In thecondition process table, a condition of an analysis result which is setin advance and is obtained by analyzing the captured image is associatedwith a process required at a position matching the condition. Forexample, it is defined in the condition process table that a processsuch as agrochemical spraying is performed when the shape, color, sizeor the like of an object (an image object of a plant) included in animage matches the condition of the pest occurrence symptom.

FIG. 6 is a diagram exemplifying a functional configuration of a remotecontrol system 1. The remote control system 1 includes a movement unit21, an image-capturing unit 22, a position detection unit 23, atransmission unit 24, a movement control unit 11, an acquisition unit12, a storage unit 13, a determination unit 14, an instruction unit 15,a history storage unit 16, and a processing unit 31. In this example,the movement unit 21, the image-capturing unit 22, the positiondetection unit 23, and the transmission unit 24 are mounted on a movingimage-capturing device 20. The movement control unit 11, the acquisitionunit 12, the storage unit 13, the determination unit 14, the instructionunit 15, and the history storage unit 16 are mounted on a remote controldevice 10, and the processing unit 31 is mounted on a control targetdevice 30.

The movement unit 21 is realized by a rotation mechanism 208 of themoving image-capturing device 20, and the image-capturing unit 22 isrealized by a camera 206 of the moving image-capturing device 20. Theposition detection unit 23 is realized by a positioning device 207 ofthe moving image-capturing device 20, and the transmission unit 24 isrealized by a communication IF 205 of the moving image-capturing device20. The movement control unit 11 is realized by a CPU 101 and acommunication IF 105 of the remote control device 10, and theacquisition unit 12 is realized by the communication IF 105 of theremote control device 10. The storage unit 13 is realized by anauxiliary storage device 104 of the remote control device 10, and thedetermination unit 14 is realized by the CPU 101 and the auxiliarystorage device 104 of the remote control device 10. The instruction unit15 is realized by the communication IF 105 of the remote control device10, and the history storage unit 16 is realized by the auxiliary storagedevice 104 of the remote control device 10.

The movement control unit 11 provides the movement unit 21 with amovement instruction including a movement route, a movement speed, analtitude at moving, and start and end times of image-capturing, in orderto control the movement of the moving image-capturing device 20. Themovement unit 21 moves the moving image-capturing device 20 inaccordance with the movement instruction of the movement control unit11. The image-capturing unit 25 captures a moving image over a periodfrom the start time to the end time of image-capturing along with themovement of the moving image-capturing device 20. The position detectionunit 23 detects a position of the moving image-capturing device 20, thatis, a position of the image to be captured. The transmission unit 24transmits, to the remote control device 10, image data indicating theimage captured by the image-capturing unit 22 and position dataindicating the position of the image.

The acquisition unit 12 acquires the image data and the position datatransmitted from the transmission unit 24. The storage unit 13 storesthe above-described condition process table, that is, information inwhich a condition related to an analysis result of the image data and aprocess to be performed by the control target device 30 are associated.The determination unit 14 analyzes the image data acquired by theacquisition unit 12, and determines whether the analysis result matchesthe condition stored in the storage unit 13. When the analysis result ofthe determination unit 14 matches the condition stored in the storageunit 13, the instruction unit 15 specifies an area to be processed basedon the position data acquired by the acquisition unit 12, and instructsthe control target device 30 to perform the process associated with thecondition in the area. The history storage unit 16 stores, as a processhistory, the process instructed by the instruction unit 15 and theposition of the area in which the process is performed. The processingunit 31 performs the process in accordance with the instruction from theinstruction unit 15.

2. OPERATION

FIG. 7 is a sequence chart exemplifying an operation of a remote controlsystem 1. In FIG. 7, a movement control unit 11 of a remote controldevice 10 provides a movement unit 21 with a movement instructionincluding a movement route, a movement speed, an altitude at moving, andstart and end times of image-capturing via a network 90 (step S11). Themovement instruction is prepared in advance by an administrator of theremote control system 1 or the like, and is input to the remote controldevice 10.

The movement unit 21 of a moving image-capturing device 20 drives arotation mechanism 208 to move the moving image-capturing device 20 inaccordance with the movement instruction of the movement control unit11. An image-capturing unit 25 captures a moving image over a periodfrom the start time to the end time of image-capturing included in themovement instruction as the moving image-capturing device 20 moves. Aposition detection unit 23 periodically (for example, every 10 seconds)detects a position of the moving image-capturing device 20, that is, aposition of an image to be captured. A transmission unit 24 transmits,to the remote control device 10 via the network 90, image dataindicating the image captured by the image-capturing unit 22 andposition data indicating the position detected by the position detectionunit 23. This transmission processing may be performed after the movingimage-capturing device 20 lands on a base facility provided at theflight end point, or may be performed during the flight of the movingimage-capturing device 20.

An acquisition unit 12 of the remote control device 10 acquires theimage data and the position data transmitted from the transmission unit24. A determination unit 14 analyzes the image data acquired by theacquisition unit 12. Specifically, the determination unit 14 analyzesthe shape, color, size or the like of an object included in each imageindicated by the image data by an image recognition technology, anddetermines whether the analysis result matches any of conditions (pestoccurrence symptom, water deficiency symptom, fertilizer deficiencysymptom, growth completion symptom, and the like) stored in a storageunit 13.

If there is an image whose analysis result matches the condition storedin the storage unit 13, an instruction unit 15 of the remote controldevice 10 specifies a position of an area, which is a processing target,based on the position data corresponding to the image. Specifically, theinstruction unit 15 sets, as the processing target, an area having apredetermined radius centering on the position data which corresponds tothe image whose analysis result matches the condition (pest occurrencesymptom, water deficiency symptom, fertilizer deficiency symptom, growthcompletion symptom, or the like) stored in the storage unit 13. Theinstruction unit 15 then reads a process (any of agrochemical spraying,watering, additional fertilization, harvesting, and the like) associatedwith the above condition from the condition process table, and transmitsa processing instruction including the processing target position dataand its process to the control target device 30 via the network 90. Thehistory storage unit 16 stores, as the process history, the processinstructed by the instruction unit 15 and the processing target positiondata of the area where the process is performed.

The processing unit 31 of the control target device 30 receives theprocessing instruction from the instruction unit 15. Then, the controltarget device 30 performs the process included in the processinginstruction in the area indicated by the processing target position dataincluded in the processing instruction. Specifically, in a case wherethe control target device 30 is capable of performing the whole processautomatically, the control target device 30 moves to the area indicatedby the processing target position data and performs the process.Further, in a case where a human assists or executes at least a part ofthe process, for example, the human refers to the processing instructionin the control target device 30, moves to the area indicated by theprocessing target position data, and operates the control target device30 in the area to perform the process.

According to the present embodiment, it is possible to determine whetheror not to perform various processes such as agricultural work withoutentrusting the determination to the human.

3. MODIFIED EXAMPLES

The present invention is not limited to the above-described embodiments,and various modified examples are possible. Several modified examplesare described below. Two or more of the following modified example maybe combined for use.

3-1. Modified Example 1

The present invention is applicable to a process of an animal or plantsuch as laver, fish, shellfish, or the like, which is cultivated in theocean or river, or a process of an animal such as livestock kept in aranch, in addition to the plant cultivated in the field.

3-2. Modified Example 2

The movement control unit 11 may control the movement of the movingimage-capturing device 20 depending on a type of an animal or plantincluded in the area to be processed. For example, when it is impossibleto analyze the image unless the moving image-capturing device 20 fliesat a low altitude depending on the type of animal or plant, the movementcontrol unit 11 controls the movement of the moving image-capturingdevice 20 to fly at a relatively low altitude. In addition, for example,when it is impossible to analyze the image unless the movement route perunit area of the moving image-capturing device 20 is dense depending onthe type of animal or plant, the movement control unit 11 controls themovement of the moving image-capturing device 20 so that the movingimage-capturing device 20 flies along a movement route in which themovement route per unit area of the moving image-capturing device 20 ismade dense. Further, for example, when it is impossible to analyze theimage unless the moving speed of the moving image-capturing device 20 isdecreased depending on the type of animal or plant, the movement controlunit 11 controls the movement of the moving image-capturing device 20 tofly at a relatively low speed.

Furthermore, in an embodiment, the determination unit 14 analyzes theimage to select a necessary process in an embodiment. Instead, forexample, a system administrator may first designate a process, and thenthe determination unit 14 may determine whether or not to perform thedesignated process by analyzing the image. In this case, the movementcontrol unit 11 may control the movement of the moving image-capturingdevice 20 in accordance with the designated process. For example, in acase where a process of watering is designated, it is sufficient toanalyze the image even if the moving image-capturing device 20 movesquickly at a relatively high altitude since the movement route per unitarea is sparse. Accordingly, the movement control unit 11 controls themoving image-capturing device 20 to perform such a movement. On theother hand, for example, in a case where a process of agrochemicalspraying is designated, the sufficient image analysis cannot beperformed unless the moving image-capturing device 20 moves slowly at arelatively low altitude since the moving route per unit area is dense.Accordingly, the movement control unit 11 controls the movingimage-capturing device 20 to perform such a movement.

3-3. Modified Example 3

The control target device 30 and the moving image-capturing device 20may be the same. That is, the moving image-capturing device 20 may beused as the control target device 30. In this case, the transmissionunit 24 of the moving image-capturing device 20 may transmit the imagedata indicating the image captured by the image-capturing unit 22 andthe position data indicating the position detected by the positiondetection unit 23 during the flight of the moving image-capturing device20. In response to this, the determination unit 14 immediately analyzesthe image data acquired by the acquisition unit 12 and determineswhether the analysis result matches the condition stored in the storageunit 13. Then, when the analysis result of the determination unit 14matches the condition stored in the storage unit 13, the instructionunit 15 instructs the control target device 30 to perform the processassociated with the above condition. This processing is performed inreal time while the moving image-capturing device 20 is flying.

3-4. Modified Example 4

The area to be processed may be an area that does not include theposition at which the image is captured, and may be an area that has apredetermined positional relationship with the position at which theimage is captured. For example, when laver is cultivated in the ocean, asign of occurrence of a red tide or the like may appear on the open seaside of the laver aquaculture area. Therefore, when the movingimage-capturing device 20 captures the open sea side of the laveraquaculture area, and moves the facility or sprays an appropriatepesticide for an area where the laver aquaculture facility is located.In this case, the area to be processed is an area that does not includethe position at which the image is captured, and corresponds to an areahaving a predetermined positional relationship with the position atwhich the image is captured.

3-5. Modified Example 5

The instruction unit 15 determines a process to be instructed to thecontrol target device 30 based on the process history stored by thehistory storage unit 16. For example, in a case where the agrochemicalis again sprayed to an area where the process history indicating thatthe agrochemical has been sprayed in the past is stored, it isconceivable an example that the instruction unit 15 instructs a processof making an amount of the agrochemical less than the previous time.

3-6. Modified Example 6

The instruction unit 15 may determine a process to be instructed to thecontrol target device 30 based on information (for example, temperature,humidity, or precipitation amount) regarding the natural environment atthe position of the image before the time when the movingimage-capturing device 20 captures the image. For example, when theprecipitation amount is extremely small before the time when the movingimage-capturing device 20 captures the image, it is conceivable anexample that the instructing unit 15 instructs the process of making theamount of watering in the area to be processed more than the usual time.

3-7. Modified Example 7

The image-capturing device in the present invention is not limited tothe moving image-capturing device 20 such as the rotary-wing aircraftexemplified in an embodiment, but may be a device mounted on a vehiclemoving on the ground, or may be a device carried by the user, such as asmartphone or a digital camera. Furthermore, the image-capturing devicein the present invention does not have to be a movable device, and maybe, for example, a fixed device having a camera which faces a knownorientation.

3-8. Other Modified Examples

A part of the functional configuration exemplified in FIG. 6 may beomitted. For example, the storage unit 13 may be provided by an externalcomputer device different from the remote control system 1. Further, thefunction assignment between the remote control device 10 and the movingimage-capturing device 20 is not limited to one exemplified in FIG. 6.In an embodiment, some of the functions implemented in the remotecontrol device 10 may be implemented in the moving device 20. Forexample, some or all of the functions of the determination unit 14 maybe implemented in the moving image-capturing device 20. In addition, acomputer device group that physically consists of a plurality of devicesmay function as the remote control device 10 in the remote controlsystem 1.

The programs executed by the CPU 101, the CPU 201, and the like may beprovided by a storage medium such as an optical disc, a magnetic disc,or a semiconductor memory, or may be downloaded via a communication linesuch as the Internet. In addition, the programs may not execute all thesteps described in an embodiment.

1. A remote control system comprising: an acquisition unit that acquiresimage data indicating an image captured by an image-capturing device andposition data indicating a position of the image; a storage unit thatstores a condition regarding an analysis result of the image data and aprocess to be performed by a control target device in association witheach other; a determination unit that analyzes the acquired image dataand determines whether the analysis result matches the stored condition;and an instruction unit that, when the analysis result matches thestored condition, instructs the control target device to perform theprocess associated with the condition in an area based on the acquiredposition data.
 2. The remote control system according to claim 1,wherein the image-capturing device is a moving device that is movable.3. The remote control system according to claim 2, further comprising amovement control unit that controls movement of the image-capturingdevice.
 4. The remote control system according to claim 3, wherein themovement control unit controls the movement of the image-capturingdevice in accordance with a type of an animal or plant included in thearea where the process is performed.
 5. The remote control systemaccording to claim 3, wherein the movement control unit controls themovement of the image-capturing device in accordance with the process,and wherein the analysis unit determines whether the analysis resultmatches the condition that is stored in association with the process. 6.The remote control system according to claim 2, wherein the controltarget device is the moving device.
 7. The remote control systemaccording to claim 1, wherein the area based on the acquired positiondata is an area that does not include the position indicated by theposition data, and is an area having a predetermined positionalrelationship with the position indicated by the position data.
 8. Theremote control system according to claim 1, further comprising a historystorage unit that stores, as a process history, the instructed processand the area where the process has been performed, wherein theinstruction unit determines a process to be instructed to the controltarget device based on the stored processing history.
 9. The remotecontrol system according to claim 1, wherein the instruction unitdetermines a process to be instructed to the control target device basedon information related to a natural environment at the position of theimage before a time when the image is captured.
 10. A remote controlmethod comprising: acquiring image data indicating an image captured byan image-capturing device and position data indicating a position of theimage; in a storage unit that stores a condition regarding an analysisresult of the image data and a process to be performed by a controltarget device in association with each other, determining whether ananalysis result of the acquired image data matches the stored condition;and when the analysis result matches the stored condition, instructingthe control target device to perform the process associated with thecondition in an area based on the acquired position data.
 11. A programfor causing one or more computers to execute: acquiring image dataindicating an image captured by an image-capturing device and positiondata indicating a position of the image; in a storage unit that stores acondition regarding an analysis result of the image data and a processto be performed by a control target device in association with eachother, determining whether an analysis result of the acquired image datamatches the stored condition; and when the analysis result matches thestored condition, instructing the control target device to perform theprocess associated with the condition in an area based on the acquiredposition data.